A Node Based Filter Editor For Inkscape:

Note

This Document is also available as an web page, with formatting the way it was intended, at https://phantomzback.github.io/GsocProposal24/ . I've authored the document as a markdown file in Obsidian, with a theme applied which I feel is a great look for documents. However, exporting this to a PDF causes significant loss of formatting and a general loss in appeal. I've done my level best to try to format the document to the best of my abilities but it still has hiccups. Therefore, I would request you to instead use the web page to read through my proposal. My humblest apologies for the inconvenience

A Gist of The Proposal

This proposal aims to be a detailed document, for my proposal for a Node Based Filter Editor for Inkscape. It contains the summary of all my research, discoveries, learnings and progress with this, and the conclusions I've drawn from them.

I hope this acts as a testimony of my abilities, and I hope I am able to convince you that I am up for this task! So here goes...

Inkscape Questions

Inkscape and I:

  1. When did you first hear about inkscape?

Inkscape and I go wayyy back. I started off programming with Scratch in 2014, when I was in class 4. I once came across Vector and Bitmap somewhere on scratch and asked my father to explain the difference, and I was amazed! Naturally, I wanted to create them. So I found out about Inkscape and started tinkering right away. It served as my starting point to getting to know about the world of Open-Source. I was hooked to this concept and found out about a lot more, GIMP, Blender, Krita, Synfig, games like 0 A.D., .... WHAT NOT! and switched my workflow to be completely open source. (All the diagrams attached here, as well as the document are made in open source software :P).

  1. What kind of drawings do you create with Inkscape?

I started out making drawings from tutorials and very basic drawings on my own, since I was young and new to the concept. Eventually, I got around to creating logos and arts inspired by the channel Logos By Nick, since they totally mesmerised me! Then, I began creating flat sprites to create animations like Kurzgesagt , and animating them using blender. (Then I tried making my own software for Vector Animations )

  1. Describe your participation in our community (e.g. uploaded drawings, tutorials, bug reports, communication via mailing lists or IRC).

I was an active member on Inkscape discord channel, until my entrance exam preparations started in 2021. Unfortunately, I remained unaware of the IRC and RocketChat channels until recently, and thus didn't communicate on the platform. But now I am an active member on the development and general channels on Inkscape's RocketChat. Since I've been an active user of Inkscape since long, I've also provided help on the Inkscape discord, and now on the RocketChat.

  1. Describe your contributions to the Inkscape development (e.g. bug fixes, translations, packaging, testing).

I have published bug fixes (this and this). I am also currently working on improvements in several areas on my local branches.

  • Creating a unified duplication system for naming when an item is duplicated: (this working on local branches to implement this locally)
  • Improving the UI of the current Filter Editor dialog:
    • Highlighting the strokes on the current editor (this)
    • Resize-able panels in the filter editor dialog (this)

A bit of an additional background: I first had a look and cloned Inkscape's source code in 2019 (I was just 14!), since I wanted to create my own SVG animation software, and I was overwhelmed looking at the source code (just like many of the developers shared in Martin Owen's videos, no doubt the source code is large). After trying for a day to try to separate the XML and Renderer from the Inkscape Source Code to use in mine, I gave up and felt it was too tough for me. Now, after having completed my college entrance exams, I have gotten around to being able to not only understand the source code, but develop patches, improvements, and hopefully, a TOTALLY NEW FEATURE!

  1. In exactly two sentences, why should we pick YOU?
  1. I love open source, I love Vector Graphics, I love Inkscape, and will work on Inkscape and try to work on features and improvements irrespective of my outcome on GSoC and will try to become a core developer.
  2. I have a strong grasp of concepts, and prior experience in application development, and I am very dedicated, which I hope would have been noticed.

About Me:

  1. What is your name, email address, and IRC nickname?

Name: Ravi Arora
Personal Email Address: yugiarora@gmail.com
College Email Address: raviarora23@iitk.ac.in
RocketChat Nickname: Ravi.Arora (aka, GarbCol05.aka.PhantomzBack, aka, GarbCol05)
Please Note: I initially joined with some aliases which caused confusion on the chat, apologies for that!
General Alias: PhantomzBack (github, gitlab, discord, etc.)

  1. Describe any plans you have for the summer in addition to GSoC (classes, thesis, job, vacation, etc.).

I would be travelling for 2-3 weeks (overlaps with the community bonding period) visiting my hometown, but can continue to work from there. Apart from that, I don't have any plans.

I can commit to a minimum of 30 hours and ideally, 40-50 hours in week. I would be available at most of the times on Inkscape's RocketChat.
I am currently located in India (UTC+5:30), and am usually available from 12 P.M. to 4 A.M. IST

  1. What programming projects have you completed?

I have worked on an innumerable number of programming projects, the most prestigious one being a Qt and CLI-based SVG Animation Software, which I designed from scratch.
It started off with me being annoyed that I couldn't find any alternative for Adobe After Effects, after discovering Kurzgesagt used AE for creating their animations. I wanted to give Adobe open source competition for this too (eventually turning my software into a layer based compositor), like Inkscape and GIMP did for Illustrator and Photoshop!
The GUI was not the best since I had to leave the project midway to focus on college entrance examinations, but it provided ability to animate elements inside SVGs, and interpolate properties.

Here's a broader collection of my projects

  • Purely C/C++ Projects:
    • Open-Source SVG Animation software - My favourite, most extensive project
    • CLI Image Processing tool, features like adjust color levels, saturation, etc.
    • Sudoku Solver
    • Tetris (using SDL2)
    • OpenCV based edge detection using C++
    • Competitive Programming Templates
    • etc.
  • Python Projects:
    • Tetris with pygame
    • Basic RPG with pygame
    • PyQt5 based printer interface app
    • Image Processing with opencv, matplotlib, in Jupyter Notebooks
    • Animation using Manim - 3Blue1Brown's mathematical-animation library
    • Music Player using web scraping (BeautifulSoup) and tKinter
    • Automated Script to Join Teams Meeting using Selenium (for online classes :P)
    • Completely self written Linear, Logistic Regression, KNN algo's from scratch
    • Discord Bots using discord.py
    • Flask based web app for a hackathon
    • Tool to render SVG's with using WebView (for my SVG Animation software)
    • Tool to render SVG's using headless chromium (for my SVG Animation Software)
  • WebDev Related:
    • Various HTML, CSS, php sites for learning (LAMP stack)
    • Various ReactJS web apps for learning.
    • A Node.js tool to export HTML to images using headless chromium
  • Arduino's and Raspberry Pi's:
    • Converting a regular printer to network printer with a Raspberry Pi
    • Raspberry Pi based cars
  • GameDev (Unity-C#, Unreal- C++):
    - Pixel Invaders - Space Shooter with an epic twist (Unity)
    - Multiplayer First Person Shooter (Both Unity and Unreal)
    - Endless runner (Unreal)
    - Clone of Bomberman (so that I could play it with my friends online :P) (Unreal)
    - Clone of Overcooked (Unreal)
    ....and a lot more
  1. What are your favourite programming tools (editor, etc.)?

Briefly,

  • Ubuntu - OS Choice
  • VSCodium - Programming
  • Obsidian - Note Taking
  • Konsole - Go To terminal
  • Python for handy tasks
  • C++ for core development, Meson and g++ for building and compiling, gdb for debugging

Reasons:
I love using VSCodium - open source VS Code bindings, due to innumerable number of extensions, the clean looks and the ease of usage. In past, I did use Emacs but didn't feel that was the perfect fit for me. I love the fact that I have features like SVG viewers, live servers, etc. The flexibility in navigating code, peeking definitions, references, etc. make it wonderful. Extensions such as Anchors, Bookmarks make it even easier to store important references and tags.
I love to make use of Obsidian for making notes wherever I need to (this document is written in Obsidian too) . But, I am considering switching to Logseq. I also keep my tab S9 handy with me to draw diagrams to think and plan out while programming.
Writing python code is easy to write and quickly written, very suitable for simple tasks.

  1. Describe any work on other open-source projects.
  • I've worked on creating my own animation software: https://github.com/MoplusplusApp. The code is a bit outdated, since I had to quit working on this due to preparation for college entrance examinations.
  • I've worked with FOSSASIA as a part of Google Code-In, on their orgmanager. I placed in Top 10 in the organisation.
  • I've also worked on testing Natron, an open-source node based compositor, and filing a few bug reports. I was an active member of their community too.
  • I've worked on Manim, I was an active member on the community and filed a few issues. I did do some bug fixes on my own but couldn't push any of them, since I had to abandon my work on it.
  1. List other GSoC projects you are applying to.

None, apart from this one on Inkscape :).

Education Information:

  • Currently, I am student, and here's my education information
  • University/College: Indian Institute of Technology, Kanpur
  • Major: Computer Science and Engineering (B.Tech)
  • Current Year: Freshman Year (First Year)

Tasks:

  1. Submit a drawing you have made with Inkscape. It doesn't have to be professional quality but it should show us that you know how to use Inkscape.
  2. Introduce yourself on the Inkscape developers mailing list, and post your project ideas for feedback, before submitting your application
  3. Checkout the Inkscape source code through git and review it. Make sure you can compile and build Inkscape on your own PC!
  4. Find and fix two bugs in the Inkscape bug tracker. Create and upload patches for the bugs, and make sure we know to connect those bug fixes and your GSoC application. You can alternatively submit a patch that improves code unrelated to a bug.
  5. Prepare a detailed plan (with timetable) of the work you want to do.

Responses:

  1. Here's an older drawing of my initials which I enjoyed making. All of my mock-ups attached in the document are designed in Inkscape too! These are tagged with #made-in-inkscape
    Pasted image 20240402213155.png
  2. Done, I've introduced myself on the RocketChat!
  3. Done!
  4. Done, I've gotten Developer tag too!
  5. The plan is the rest of the proposal! Timeline attached too!

Need for a Node Based Filter Editor:

Problems with the Current One:

#ux
The current filter editor, is a layer based filter editor, with various drawbacks, and is not very user friendly. It is not very intuitive to use, and doesn't provide great user-experience, severely slowing them down.

  • The filter editor, is highly capable of creating some wonderful effects, but the UI is extremely non-intuitive for beginners, and drives them away. The current editor makes it really hard to understand what's going on.

  • For starters, the ease to work with decreases significantly as we increase the number of primitive effects in the filter. It is fairly annoying to understand the connections in the current format and the connections get seemingly more and more frustrating. Here's an example of what the editor looks like after applying just two filters from the Filter Gallery.Pasted image 20240320013323.png

  • Users are limited to viewing parameters of only one primitive filter at a time. This creates an unsatisfactory user experience, since quite often, users would wish for ease in modifying parameters of multiple filters at a time, to get the desired look they wish to achieve. Doing so right now would be a tedious task, but this would be wonderfully resolved with the help of a Node-based workflow.

  • Making connections between various filters, or the source channels, has no highlighting, and is not tactile.

  • There's no simple way to preview the effects of a filter up-to some primitive filters. There are workarounds for this, such as using a placeholder merge node just and plug-in what we wish to see into it, but such methods are undesirable. However, this can be easily integrated in a node based editor by giving the option of providing a preview up-to each filter.

  • The primitive filter editor gets very cluttered, which can easily be avoided with node groups, similar to the functionality offered in blender.

Implementation Plan:

The plan has been broken up into various segments, I have also added created tags for the reader/reviewer to be able to easily explore parts relevant to them, feel free to make use of them. Although, it is recommended to go through all of it to understand the complete flow and plan, with no stone left unturned, and since their is considerable overlap of topics at various places. This segment also consists of my research and conclusions.
The tags for navigation are:
#technical - Targeting development
#ux - Targeting UI/UX discussions
#important - Do go through these.

To Begin With:

#important #technical #ux
I have put in a lot of hours into refining and doing my best to perfect my proposal and application for this project, and giving a clear and detailed analysis to reflect on my abilities to take on such a task. To accomplish this, I made it my goal to approach this in a systematic manner and with a clear vision, and start the work by building a clear foundation, and not hastily rushing into getting started with the development, to also support future prospects by creating re-usable functionality by making abstracted and templated classes.

  • #ux To get started, I put in great efforts into analysing the existing editor, finding out flaws in the current one, taking reviews on problems faced by users in the current one, figuring out whether a node editor would really be the best replacement for a filter editor, and if so, what should be the design? I analysed various existing node editor ideas to try and come up with a strategy fit for our use case (listed below in UI/UX analysis). During this time, I also focused greatly on connecting with the community, users as well as developers to try to match the overall goal. I created mock-ups after discussing with various members on the group, I researched various existing node editors to come up with ideas for how to design it to be a good fit in Inkscape. I had constant healthy discussions with various members, took polls and reviews on Reddit and the Inkscape discord, and Rocket Chat, about features lacking in the current one, and what they really needed, and spent time modelling the designs and concepts.

  • #technical Rather than putting my focus on creating a prototype, I shifted my focus on figuring out the Best Strategy for implementation. I researched on libraries which offer functionality like a Node-based editor, libgtkflow and gtknodes, and each of these had their fair set of pro's and con's. A detailed analysis is given in the appendix (libgtkflow vs gtknodes). I concluded that it is a better decision to not stick to either of these. I also tried to find various GTK based software to try to figure out if any have a node based editor which we could possibly extract, or take inspiration from, but unfortunately, couldn't find any. So, I concluded that it's best to create a Node based Editor from scratch, taking inspiration from them. Each part of it tailored to fit Inkscape, abstracted and implemented the way we want it, and set up a basis for future similar editors in the future. And it was also necessary to see whether going along the GTK Widgets approach, rather than a custom drawing area would be a better approach. For example, in my research, I understood that zoom is definitely not an intuitive thing to tackle when working with Gtk Layouts, Windows, etc.. And this would be a deal breaker, since everyone's display has varying DPI, it's highly highly inconvenient to constantly keep resizing widgets or be forced to one view.

  • After doing my clear ground research, I implemented a prototype. It is discussed ahead.

UI/UX Analysis and Mock-ups:

  • Since currently, it is not possible to dock anything underneath the canvas, the filter editor would be developed as a dialog, replacing the current one.
  • In most working scenarios, the user would have a portrait-ish dialog, i.e., the vertical allocation would be more than the horizontal, hence, a top-down workflow is more important than a left-right one.
    Users Typical Working Area:Pasted image 20240401034843.png
  • Filters are all about getting the right effect you are going for, much like creating materials, procedural shaders, etc. and require constant tinkering. Therefore, it is essential to allow users to view parameters related to multiple filter primitives at once. Additionally, it is a good option to allow users to fold/minimise the contents of a filter, to not consume screen space, as well as for comfort for the user.
  • While working with filters, it's sometimes helpful to label filters according to the purpose they serve, to prevent confusions when working with a larger set of filters. (An example is showcased in the mock-ups)

Mock-ups:

Here are the mock-ups I have designed while discussing with development team, as well as UX team. I am still working on coming up with more, and would be showcasing them on the relevant channels soon! I've been a bit mischevious with the themes and color choices, and tried coming up with fresh ideas rather than solely matching Inkscape's design, but this was just for me to try out. The final mock-ups would fit in well with Inkscape's theme
These are designed after taking inspiration from various Node based editors (Blender's Material Editor, Nodebox's Node Based Editor, Enso's node based editor)

  1. Top Down Workflow with Parameters in Canvas: Pasted image 20240401034401.png

A More Complex Node:
Pasted image 20240401035328.png
Pros:

  • Focus on having a clean top-down workflow
  • Giving the user his comfort by ensuring nodes can be toggle minimised
  • Allow user to modify properties of multiple primitives at the same time
  • Allow user to rename nodes

Cons:

  • The widgets for properties are small, and would require zooming in at times to allow better handling.
  1. Top Down Workflow with Separate Properties Panel: Pasted image 20240401035445.png

Pros:

  • Clean top down workflow with no messiness ever, due to separate properties panel

Cons:

  • Can't edit properties of multiple primitives at the same time.

#future An Illustration for how Preview can be incorporated:
Pasted image 20240401035845.png
A fold-able panel for each node, allowing the user to toggle it for each node.
This allows the user to be able to preview the output upto multiple filters simultaneously, and also allow the user to see the effect applied by an individual filter.

Prototype

A prototype meant to showcase what you can expect and prove my ability to be able to take on this task. A video showcasing the various elements of the prototype is available here (contains audio cues)

An image of the Prototype in action:
Pasted image 20240401230047.png
Works With Zoom: Pasted image 20240402201258.png

The prototype features:

  • An Infinite Canvas
    • Fold-able nodes containing placeholder widgets (1)
    • Context menu to add and delete nodes
    • Smooth Scrolling, Smooth Panning
    • Multiple Selection with Rubber-band Selection, Modifiers (2)
    • Bringing the selected widgets on top
    • Ability to move around selection and place anywhere on canvas
    • Well handled, responsive connections
      • Rendered behind nodes to not interfere with work.
      • Ability to:
        • Make multiple connections from a single source (3)
        • Make connections starting from sink
        • Override connections on sink by plugging in a new source
        • Easily break/destroy connections.
      • Highlighted connections when nodes corresponding to connection are selected (2)

Additionally, the behaviour also works with zoom, but the code for that was incomplete and hence transferred to another branch.

Guaranteed Deliverables:

Functional Node Based Filter Effects Editor with the following Features:

From the UI/UX perspective: #ux

  • Functional Node based Filter Editor replacing all the existing Filter Editor functionality, in a well thought, intuitive, user friendly, with some of the key features higlighted below:

  • Node Editor Canvas:

    • Functional node-based editor with:
      • Infinite Canvas:
        • Scrolling, Panning, Zooming
        • Relevant Context Menus
        • Selection Behaviours
      • Nodes, Sinks, Sources and Connections:
        • Well implemented system for handling nodes, sinks, sources and connections
        • Intuitive connection handling
    • Details discussed in timeline and Canvas
  • Filters System And Integration With Document:

    • Nodes relevant to each filter primitive, with corresponding, intuitive widgets
    • Ability to connect filter primitives in an intuitive sense and create meaningful filters, which is reflected in the document and on the canvas
    • Details discussed in timeline and Filter System and Integration with Document
  • Preserving the Node Editors State:

    • Being able to load up the node editor the way it was when you closed it
    • Auto-formatting the nodes whenever required
    • Details Discussed in Node Editor State

Optional Deliverables (if Time Permits):

This lists out some implementations I can add to the editor, if time permits, although some of these might be overly ambitious for now! The tag #future highlights these at other places.

  • Node Groups: #future
    • Ability to group nodes and create a single node out of those, to de-clutter graphs, as well as reuse the same set of nodes for a particular functionality. (This should not be to hard to implement either, I will try to work on this if time permits)
  • Preview: #future
    • Creating preview nodes which allow user to preview the effect of a single node alone, or the chain up-to it, by intercepting data from rendering pipeline! (This is relatively tough since it requires calls to the Rendering API, while also ensuring caching and minimum calls to improve performance and make it robus)
  • Sink Source: #future
    • Creating Widgets that can act as a Sink for Sources, and Sources for Sinks, to clear up graphs and create clean connections, similar to blender (This should not be hard to implement either and mostly will be implemented as a part of GSoC)
  • Minimap: #future
    • A minified view of the bounds of the canvas. (This is not too hard either, but it is not as important of a feature as a few mentioned above. It's implemented in libgtkflow and the code there can be used for reference)
  • Creating a tutorial series for filters: #future
    • Filters are extremely powerful for creating various effects. A series explaining their usage would greatly help users benefit in being able to develop better art. I plan on working on this as soon as the filter editor is stable and ready, so that users can learn and create art with the newly developed editor
  • And any other features recommended and discussed by community :)

Timeline:

May 1 - May 26:
Community Bonding Period:

  • Get to know mentor, discuss the project with them, along with various aspects of the project, seek guidance in areas where required, discuss project guidelines and overall guidelines.
  • Get to know the members better, spend time working with the development and UX team, to come up with a conclusive, finalised UI/UX mock-up as well as discuss other technical specifications.
  • Spend time on my own understanding source code better, try to create documentation for undocumented parts of the filter editor that aren't likely to change. With the current plan, these would include:
    • Work on documentation for Filter Editor and improve understanding of it again
    • Understanding relevant lib2geom classes, and how they can be used to handle geometrical options of the canvas to simplify geometry management. This also serves to ensure uniformity in source code, rather than coming up with new classes for geometry management for the canvas.
    • Get more familiar with coding style - A great take back for me would be to learn to write code according to standards and in a manner that the code can be understood and appreciated, or briefly, write neat code.
    • Going through testfiles and resources on Testing to get familiar with how to create effective tests, get an idea of how to develop a Unit Test.
  • Study the Wiki! The wiki has a lot of relevant information for beginning development- the Vision of Inkscape, writing desirable code, preferred conventions, tips and tutorials on topics such as safe memory management, etc. These give a lot of insight for getting started with development while following best practices, something I wish to learn very much!
  • Coming up with the final draft, having a clear picture in mind for beginning the implementation tasks!

May 27:
Coding Period Begins:

Note

I've condensed the completion of tasks into a period where I can implement them comfortably, but often, things go wrong and take unexpected turns. In light of this, I've kept time in the last few weeks as buffer to catch up in case of such events, and kept light tasks towards the end. Needless to say, I would always try to be running ahead of the clock! Also, some of these are bound to change to some extent based on conclusions drawn from further discussions. Additionally, the GTK4 branch's filter editor is currently broken. Until it is recovered to a stable state (I will try contributing in this), I may work on the 1.4.x branch and forward port the code later.

#prototype are features I have already showcased in my prototype

#research are features/tasks which would require a significant amount of time for me to think and research before being able to tackle (please don't get me wrong, I would be requiring research and thinking for every task, but tasks tagged with this would require a major amount of research first, because I have made little to no progress in understanding these and planning how I would implement these, or lack any idea of implementing this at all)

Important

Please Note: Mostly, if there are no changes in the technical specifications and implementation method, implementing the canvas can begin from within Inkscape, due to the nature of my plan, since these widgets can safely replace the widgets in the current filter editors dialog, making the integration into Inkscape easier. Beginning from within Inkscape is also probably a better approach, so that event handling (context menu, clicks, modifiers, hotkeys, etc.) can be inherited the way Inkscape does it, rather than creating controllers for event handling in a dummy app, and then putting in more efforts to port it into Inkscape. So, I would like to start my development from within Inkscape!

Week 1: May 27 to June 2, Week 2: June 3 to June 9

  • Start Working on Canvas, Nodes, Connections, etc.:
    • Implement these Features:
      • Base Classes for Infinite Canvas
      • Base Classes for Nodes
      • Base Classes for Sinks, Sources and Connections
      • Basic Canvas Operations:
        • Scrolling, Panning, Zooming #prototype
        • Context Menu
          • Add Nodes #prototype - Search bar for ease
          • Delete Selection
          • Move Selection #prototype
          • Auto Layout
          • Other features discussed with users
        • Clipboard and Clipboard Operations #research
          • The usual ones: Cut, Copy, Paste, Duplicate
      • Basic Node Behaviours:
        • Selections
          • Multiple Select with Modifiers #prototype
          • Select All, Select Connected Chain
          • Rubber-band Selection #prototype
          • Move Selection with behaviour on edges handled
      • Sinks, Sources, Connections:
      • Ensure styling matches agreed UI mock-up
  • Integrate these into new FilterEffectsDialog to have a placebo node editor in place

This would handle the integration in terms of bringing the widgets to Inkscape. The integration of connecting these widgets to filters in a functional manner is explained ahead.

Week 3: June 10 to June 16, Week 4: June 17 to June 23

  • Begin Work on Filters System
    • Develop layouts, widgets, for each of the Filter Primitives, with the widgets for their corresponding attributes:
      • Majority of the widgets, which shall be used for various attributes, already have their classes developed, (such as SpinButtonAttr, MatrixAttr, EntryAttr, FileOrElementChooser, etc.
      • The class settings contains the layout for each filter primitive type, this can be used as reference for the alternative for the same in the new editor.
      • Ensuring planned out layouts, according to mock-up's, taking extra time and precautions to ensure no buggy or unexpected behaviours which could be destructive to user's workflow.
      • Incorporating styling into the work done till present, and smooth behaviour
      • Incorporating region settings, etc.
  • Buffer Time - To work on feedback received on implementation till now, resolving bugs, etc., else begin work ahead

Week 5: June 24 to June 30, extends into Week 6

  • Filters System and Integration with Inkscape's Document:
    • Mapping each filter primitives's layout and Attribute Widgets (AttrWidget) to the corresponding primitives (if a separate parameter dialog is not used, else, mapping the layouts onto the parameter handling widget)
    • Setting up the context menu to allow Adding Nodes of various types (maybe with a search dialog) ( #research going through source code of Inkscape to correctly understand the functioning of context menus within Inkscape)
    • Connecting Canvas Events with Document Changes:
      • Adding primitives, connecting primitives, handling loose primitives (which aren't connected)
      • Correctly mapping these signals emitted on editors event changes to respective SPFilter slots
      • Also mapping signals for changes in primitives to reflect
      • Goal is ensuring any change done is reflected in the document in the way it is supposed to be done.

Week 6: July 1 to July 7

  • Carried over work from previous week
  • Writing Unit Tests for parts covered till now: #research
    • Creating tests for methods that are used for functionality in the canvas (maybe to exercise them thoroughly and try to discover memory leaks, segmentation faults, I would love to learn more about how to write tests, and what they should target).
    • Creating tests on filter system by emitting signals and saving documents to see if the document is updated the way it should be
    • Note: The above unit tests may change with time, especially the one on filter system, since after integrating the Node Management System, the way the document is saved would change.
  • Working towards submission for Midterm Evaluation

Midterm Evaluation: July 8 to July 12

  • Expected Deliverables:
    • Functional Node Editor with features listed above (features are subject to changes if specifications for the editor are changed)
    • Unit Tests Covering Canvas, and Filter Work Done till now

Week 7: July 8 to July 14, extends into week 8

  • Integration into Inkscape:
    • Implementing Node Editor State Management System:
      • Implementing creation of Create, Load and Write state
      • Loading state onto Node Editor canvas when constructed.
      • Refreshing canvas for each filter

Week 8: July 15 to July 21

  • Implementing Undo System #research
    • Requires some more research and thinking, since I was unable to understand the undo part completely (does merely calling DocumentUndo::done suffice? Need to understand this functionality a lot better)
  • Buffer time to work on resolving bugs, working on feedback provided on previous work, manual testing to discover vulnerabilities, broken functionalities

Week 9: July 22 to July 28, Week 10: July 29 to August 4

  • Working on Optimisation to further improve performance, wherever possible
    • This is necessary, so that users always have a responsive editor to work with, and since the application is large, various aspects require various computational resources, thus it's important that the editor uses up minimal possible resources.
    • Currently, I can think of optimisations in a few forms:
      • Fewer redraw calls on canvas by using caching wherever possible, or separating drawing areas from widget areas to handle fewer global redraw calls.
  • Working on abstracting, creating templates and organising classes to facilitate implementation of similar dialogs/editors in future for various purposes and act as base classes for Graph based editors, as well as documentation for reference when re-using.

Week 11: August 5 to August 11, Week 12: August 12 to August 18

  • Continuing to resolve any issues
  • Updating Documentation for wherever it was missed, cleaning up code, final checks
  • Video highlighting features of the tool
  • Writing smarter tests focused on the goals outlined in Quality Assurance Initiative
  • Working towards the final test: merging into Inkscape!!!

Final Week: August 19 to August 26:

  • Submitting Final Evaluation
  • Get the Node based Filter Editor Incorporated into Inkscape!

Technical and UX Details for Implementation:

There are two broad approaches
#technical

  • Implementing on a canvas using DrawingArea or GLArea
    • This approach would be very tedious due to having to rewrite all widgets, behaviours, etc. since there seem to be no good libraries, which provide canvas widgets, and DrawingArea and GLArea do not directly supporting placing widgets on top of them. They are basically plain widgets which give us easier drawing API's, the same effect can be reproduced on any other widgets by overriding snapshot-vfunc in GTK4 and OnDrawSignal in GTK3.
  • Implementing a canvas with support for adding widgets and moving them around freely.
    • This is the more practical approach since to extend the node editor for other purposes in future, having the ability to use widgets implemented within the editor, without having to re-implement each. Custom drawings can be made over these too!

Firstly, reusing the code!
A lot of behaviour we are looking for has already been implemented, such as,

  • The CANVAS => The way the canvas is implemented gives a great starting point to getting started with our node editor canvas. The way it is implemented allows us to create an infinite, smooth scrolling canvas. Using the code from the classes, and the classes offered by lib2geom (to be used in the same way it is for the canvas )
    • canvas_grid.cpp - contains the logic for the scrollbars, adjustments, as well as their signals (specifically). The deskarea, _canvas->get_area_world(), viewbox use Geom::Rect to calculate the bounds of the allocated canvas. The same can be done for the widgets canvas, except using recursive Gtk::Viewport and Gtk::Fixed's to handle this. The complete approach is attached in the canvas section below.
  • Click behaviour: Figuring out which node has been clicked, unselect on another click, etc.
  • Drag and Drop behaviour
  • Nodes behave the same way (in some aspects) as SVG rectangles (or in general, the way SVG elements currently behave in the main canvas as well as XML)?
    • SVG rectangle elements store, x, y, width, height, and a node should preserve the same, and can be stored easily in the corresponding filter's XML
    • Other attributes of Nodes would be the filter parameters in the XML
    • They can be dragged and moved around, just like rectangles in the Inkscape canvas :)
  • Current Filter Editor Dialog: The current filter editor dialog shows how the classes such asSPFilterPrimitive and SPFilter are used, how the editor is setup to reflect changes in the editor with changes in the document, how the current UI is wired, etc. This provides great insight in the approach to use while replacing the filter editor.

Canvas: An Infinite Canvas

After researching and testing out the available classes from GTK, I came up with two approaches for implementing an infinite canvas with support for panning, zooming, etc.

Approach 1: Based on Scrolled Window
The prototype is based on this approach.
Rough Class Structure:

  • canvas_sw : Gtk::ScrolledWindow

    • canvas_viewport : Gtk::Viewport
      • canvas_outer : Gtk::Fixed
        • canvas_inner : Gtk::Fixed
  • Viewport gives us adjustments which we can as well as the ability to place our child widgets as we wish (this could be replaced with a custom layout manager too, or a Gtk::Fixed to place outer and manage it's placement, which would require us to re-implement some functionality, such as the adjustments, and handling their update signals)

  • We use inner for having the ability to place our widgets wherever we wish inside the canvas area. This gives us control over being able to place our nodes freely.

  • outer -> this may seem unnecessary, but outer is crucial for implementing zoom, since implementing zoom is probably one of the hardest and non-intuitive parts of this, since GTK offers no native support from zoom. I plan on implementing zoom using CSS transform: scale(). However, due to a prevailing inconsistency in GTK's source code, transform: scale(2) does not update the widgets dimensions correctly, and instead causes it to overflow. So, we use an outer window, which adjusts it's size to be able to contain the overflow, when we capture a zooom event from the user.

  • Drawing of connections is taken care of on Cairo contexts by overriding the snapshot-vfunc in GTK4 and on_draw in GTK3.

  • Event handling is taken care of by events received on canvas_inner, it is responsible for recognizing clicks, drags, etc. and taking consequent actions depending on the target

  • For cleaner implementation, we can override allocate and size-measure functions

  • The approach uses two coordinate systems, the reason should be clear below.

Schematics For Handling:

Note

Here, viewport refers to the region visible within the bounds of the scrolled window container.

Default view (When created or when True Bounds lie within the scope of viewport):

Pasted image 20240402222404.png
#made-in-inkscape
True Bounds is the minimum size request that inner would have.

When Panned:

Pasted image 20240402222424.png
#made-in-inkscape
Both outer and inner adjust their sizes (same when no zoom) to to accommodate the scroll.

When Zoomed:

Pasted image 20240402222805.png
#made-in-inkscape
The widget inner overflows from it's true allocation, we adjust the size of outer to so that the overflow can be captured in the scroll!

  • Setting up virtual coordinate system to handle positions of objects globally, and maintain the relation between the virtual coordinate system and the coordinate system with respect to the viewport/scrolled-window always.

  • Click and Drag behaviour: Implemented in the Prototype. The canvas widget maintains list of references to selected nodes, and un-selected nodes, and handles behaviours with clicks along with modifiers (Shift + Click adds to selection, Click clears selection and selects the new node, or un-selects it if it was the only node selected, etc.). Selections can also be dragged when any selected object is dragged with the cursor.

    Note

    Note: This will first ask the child widget whether the click was on a Node Source/Sink and decide if a connection needs to be drawn or the selection needs to be moved.

  • Rubber-band Selection: Drag select on inner, find the widgets inside the selection either using Gtk provided methods (Gtk::Widget::intersect or cairo_region_intersect or manually check the widgets with bounds in this area), but a better approach would be to do the geometry management using Geom::Rect due to the ease of operators, and for uniformity in source code.

Approach 2:

  • This approach uses recursive fixed objects along with custom scrollbars to achieve an outcome similar to the above. We virtually create our own scrolled window by using a Fixed and our own scrollbars to accordingly place the classes outer and inner and writing our own update functions for our scrollbars, or alternatively, writing our own widget which acts as a container for outer and inner and creating our own size-allocate and measure functions, along with scrollbars and logic for updating them.
  • We may be able to avoid the need for outer since it may be possible to allow viewing the overflow using the custom size-allocate function

The final approach to use will be decided after further testing, and is subject to changes.

Nodes:

The rough class structure

  • filter_node : Gtk::Box/Gtk::Overlay
    • filter_node_data_container : Gtk::Box
      • labels and fold buttons
      • Layout for filter specific widgets
    • sinks and sources

What they do

  • filter_node: helps with overall management in placing the nodes on the canvas, and acting as a container for the rest of the node interactions, and well managed coordinate system offsetting from 0.
  • filter_node_data_container: Parameter widgets, their respective layouts and layout managers are contained in this. This would also contain the styling for the node, if any.
  • Filter specific widgets: These widgets are specific to the filter type.
  • sinks and sources: Widgets which are a part of filter_node. Capturing clicks on these can convey/signal to the canvas to start creating a connection, etc. Since the click handling in the prototype is handled purely by the canvas, and the canvas further uses Gtk::Widget::pick, the canvas widgets can easily handle clicks on these and start creating connections, and store connections according to whether the drop was successful or failed
  • Basic Behaviours:
    • Collapsible Behaviour #prototype
    • Drag and drop ability on canvas #prototype
    • Drag to an end moves the canvas in that direction
    • Node Stack: Bring a node to top if it selected #prototype
    • Handling making, breaking connections. #prototype
    • Redrawing Connections upon movement #prototype
  • Ability to add children widgets #prototype

Connections:

Cairo beziers, which should ideally to be drawn under the widgets (shown in prototype)

  • The reason for this is connections on top can screw up the layout, as shown in:
    Pasted image 20240320183841.png
    [Reference: libgtkflow]

  • Drag behaviours: #prototype

    • Starting from Source, ending at Sink
      • New Connection if the sink doesn't have any Connection
      • Delete the Old connection at Sink and Replace with new one
      • Allow multiple connections from same source
    • Starting from Sink, ending at Source
      • New Connection if the Sink doesn't have any Connection
      • Delete the old connection and replace with new one
  • #ux : Clicking on source and then Modifier+Click, such as Ctrl+Click could be used to complete a connection rather than having to drag

  • #ux #future: Create sink-source style widgets, which act as sink and source, to help clean up the graph, similar to blender.

Here's the current Look in the prototypes:
Pasted image 20240402024519.png

Auto Layout

  • Coming up with a robust algorithm for automatically arranging nodes in a neat manner, with suitable placement of nodes, in proportion to node sizes, and avoiding overlap of connections to the maximum extent possible.
  • Rough Logic:
    • Define a suitable minimum height between each level of nodes, a level here refers to nodes equidistant from the node beginning the chain.
    • This would be the height between levels along the longest chain connecting the topmost node - the Original Source, and the bottom-most node - Filter Output node.
    • Define a suitable minimum width between two objects having the same parent, or at the same level along a chain/sub-chain of nodes. This would be the width between two nodes on the same level, which either converge into a single node, or don't diverge into separate sub-chains.
  • Will be useful when implementing node editor state too

Schematic Diagram For The Logic of Auto-Layout: #made-in-inkscapePasted image 20240402221859.png

Filter System and Integration with Document:

  • Involves bridging the gap between creating graphs in the node editor and creating filters in the documents
  • Re-use code available in filter-effects-dialog.cpp, since it contains the code used for the current filter editor
  • Connecting creation of filter with canvas construction, creation of nodes in canvas with creation of filter primitives, updating primitives based on the changes in their respective nodes, etc.
    • A lot of functionality in terms of the methods such as FilterEffectsDialog::add_primitive(), need not be re-written, these can be used and connected with signals set up from the Node Editor Canvas
    • The classes SPFilter, SPFilterPrimitive suffice in what we need, we need to only link canvas events with their respective creation, deletion, updates, etc.
      • Relevant methods and slots are:
        • SPFilter::child_added
        • SPFilter::remove_child
        • SPFilter::order_changed, etc.
      • The main task lies in connecting these with our canvas events, the implementation of handling the document part of this is mainly done in the existing filter editor.
  • Some more information in Appendix 2: 2. My Understanding of the Filter Editor and Filters in Inkscape

Node Editor State:

#important #ux #technical
By Node Editor State, I refer to orientation, positions, etc. of node's, and the current view of the editor. This feature is Essential since it's important that a user is able to restore his node editor after re-opening his file, or load a filter not created in the node editor, onto it, in a readable manner.
It's essential that the node editor is able to handle:

  • Creation of A Node Editor State:
    • When a new Filter is created
    • When a file with a filter is opened which doesn't contain the metadata for the Node Editor (Auto Layout)
    • When the metadata is corrupt, or incomplete. (Auto Layout)
  • Reading A Node Editor State:
    • When a file is opened and it's filter is to be loaded into the editor
  • Saving the Node Editor State:
    • Writing the metadata to the document such that it can be re-opened later on.

Creating Node Editor State:

  • When there's no existing data, creating it from scratch.
  • When there's already data in the filter editor, using Auto Layout.

Saving Node Editor State:

  • Possibly connect each node to it's corresponding filter primitive in the SVG filter data, and store the coordinates of the node in attributes like inkscape:node:x,inkscape:node:y of the filter primitive (<fePrimitive> for the corresponding node), other attributes such as toggle visibility in inkscape:node:toggle-visibility (which is removed in the plain SVG).
  • Metadata about the current view of the canvas (defined by coordinates, zoom), stored in the attributes of the corresponding <filter> tag.
Note

Implementation Note: These are to be registered in the file attributes.h under SPFilter and Filter Primitive Commons

Reading Node Editor State:

  • If metadata is complete, and not corrupted:
    • Create canvas and nodes
    • Configure and place nodes according to their metadata attributes
    • Match canvas view according to metadata
  • If it is incomplete, corrupted, or absent:
    • Create new state using Auto Layout

This would require registering the newly chosen attributes in the attributes.h file, then modifying the build , write, etc. methods of SPFilter and SPFilterPrimitive to incorporate the attributes we create, relevant to the state. Using these methods, we then construct the canvas, with their respective nodes and their placement, etc.

Conclusion:

Here's all my research and planning summarised and broken down. I know it's a bit messy, I struggle with being able to organise my thoughts clearly. But, I hope in all this chaos, I have been able to prove my capability in taking on this task.

Irrespective of whether or not my proposal is accepted or not, I would love to spend my summer working on Inkscape and contributing to it's development to the best of my abilities, and learn a lot from the teams, and the members of the community!

*Hope to enjoy a wonderful summer with Inkscape and Inkscape community ahead
Signing Off,
Ravi Arora

Appendix:

1.libgtkflow vs gtknodes

Important

An important thing to note about both these libraries is these have implementations with primary purpose (or at least, one of the primary purposes) of allowing data-exchange between nodes by setting up signals-sockets targeted for data-transmission. For majority of our use of the node-editor at present, we don't need data-transmission between nodes, as much as setting up the correct signals with the correct slots to manage connections between nodes, their is no use of transmission of any data beyond this, since the editor only sets up the order in which filter primitives are connected and their respective properties, making this feature redundant. From my understanding, we don't need to communicate this data between nodes since there's no flow of data we are trying to achieve, it's more about specifying the order in which the nodes (representing the primitives) are connected and each node should be concerned with just it's own priorities. Data flow would rather be needed for a purpose such as if each node actually processed an image, and then showed it, and then passed it on to the next one. So it might seem like data-flow may be needed for a preview node then (to be implemented in future), I don't think it would be so since we fetch the data from the rendering pipeline and the filter editor would have no role in communicating that data between nodes. But in any case, it can be implemented with ease too!

Advantages of libgtkflow:

  • Support for GTK4
  • Author can be contacted easily.
    Disadvantages of libgtkflow:
  • Code is in Vala, bindings for C++ need to be written manually using gtkmm.
  • Lot of broken functionality, in terms of UI, rendering, canvas actions, etc. such as widgets placement, connections go above widgets.
    Pasted image 20240320183841.png
  • No infinite canvas, and it uses GtkWindows and GtkWidget's rather than drawing canvases, but with poor handling.
  • Canvas resize logic is poor and inconvenient

Advantages of gtknodes:

  • More stable UI
  • Better Design
    Disadvantages of gtknodes:
  • GTK+3 based, requires manual porting to gtkmm4, which is a tougher task since it uses deprecated and removed API's.

2. My Understanding of the Filter Editor and Filters in Inkscape

  • The source for the Dialog for Filter Effects is stored in filter-effects-dialog.cpp
    • SPFilterPrimitive and SPFilter act as the document nodes for filter and filter primitives in the document.
      • These are CRUCIAL and are essentially the basis for bridging the gap between documents and the editor
    • _filter_modifier - Contains list of filters
    • _primitive_list - Contains the list of primitives contained in the current filter
      • on_draw_signal used to draw the current UI of the filter editor
      • snapshot_vfunc in GTK4
    • _settings - Responsible for handling the parameters of the UI related to the current filter
  • filter-chemistry.cpp contains the various methods for constructing, removing, filters, adding primitives, setting their blend modes etc.
  • filter-enums.cpp contains the enums for filters, along with their string representations in SVGs and utility objects for their conversions
  • filter-effects-chooser.cpp - handling of the widget and corresponding operations on the filter primitives for Blend Mode, Opacity, etc.
  • Rendering for filters is handled within the files nr-filter.cpp , nr-filter-primitive.cpp, etc.
    • To implement a preview node, the rough logic is:
      • Modifying the Filter::render function on our own, passing our own resolution to it and intercepting the surface and getting the image data, for each filter.
  • filter.cpp - management and methods for handling the pre-made filters
  • filter-all.cpp - contains the pre-made filters available in the filters drop-down.